Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions

We propose a new type of neural networks, Kronecker networks (KNNs), that form general framework for with adaptive activation functions. KNNs employ the product, which provides an efficient way constructing very wide network while keeping number parameters low. Our theoretical analysis reveals under suitable conditions, induce faster decay loss than by feed-forward networks. This is also empirically verified through set computational examples. Furthermore, certain technical assumptions, we establish global convergence gradient descent KNNs. As specific case, Rowdy function designed to get rid any saturation region injecting sinusoidal fluctuations, include trainable parameters. The proposed can be employed in architecture like Recurrent Convolutional etc. effectiveness demonstrated various experiments including approximation using solution inference partial differential equations physics-informed and standard deep learning benchmark problems convolutional fully-connected